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ABSTRACT 

This study compares student exam performance of undergraduate students in classroom and online course 
sections with nearly identical instructional designs. The sections differed only in lecturing and peer 
discussion activities, which are typical differences of classroom and online instruction. Classroom 
discussion activities included synchronous speech, while online discussions used asynchronous text. 
Composite mean exam scores show a large effect size difference that is statistically significant. Results 
suggest asynchronous peer-to-peer discussion is more effective than traditional classroom lecture- 
discussion for undergraduate students. 


I. INTRODUCTION 

Online instruction, also called web-based instruction, is increasingly used in higher education and K-12 
institutions (Allen & Seaman, 2011). The continuing rapid increase in online instruction raises several 
important questions. Is this trend beneficial for formal learning? How well do students learn in online 
instructional environments compared to classrooms? What differences between the environments affect 
instruction and student learning performance, and why do these differences matter? 

The goal of this study was to determine if students performed differently on exams given in nearly 
identical courses delivered online and in a classroom. Further, if differences existed, this analysis sought 
to examine potential causes of the differences in student performance. While the courses were delivered 
via different instructional environments, much of the instructional design of the online and classroom 
classes was identical. 

To understand potential differences between instructional environments that might affect student learning, 
some concepts must be clearly defined. First, a communication medium is a technological means to store 
and/or transmit information. Instructional methods are procedures and techniques intended to promote 
learning of specified outcomes. Instructional methods comprise three functions: presentation of 
information, specification of learning activities, and assessment. Learning activities go beyond merely 
presenting information by having students express their knowledge in ways intended to support 
achievement of learning objectives. Instructional delivery environments, such as classroom and online 
environments include various media, tools, and functions that support instructional methods. 

Using a specific instructional environment does not require the use of specific instructional methods. 
Online and classroom environments are sufficiently flexible to support a wide range of similar methods. 
While the two instructional environments can be equivalent in learning effectiveness—since they use 
similar instructional methods—the environments often differ in preferred methods. Each instructional 
delivery environment has affordances that guide selection of media and methods. Some methods are 
easier to implement and/or execute within each environment, and these methods are the ones that tend to 
be used. Specific designs and design elements are encouraged by aspects of the environment because of 
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the efficiency of implementation and use of those designs. Different instructional methods, therefore, tend 
to be associated with classroom and online environments. This report focuses on what aspects of preferred 
instructional methods affect learning, as measured by classroom exam performance. 

II. THEORETICAL FRAMEWORK 

Information internalization and extemalization by students are important processes for learning (Cress & 
Kimmerie, 2008). Internalization occurs when students encounter new information and process that 
information into knowledge. Students also learn by expressing their knowledge as information. The 
process of transforming knowledge into external information results in reinforcement of existing 
knowledge and the creation of new knowledge by increasing associations among internal representations. 
Information processing models of learning identify elaborative rehearsal as a key control process by 
which newly-experienced information is merged with existing knowledge in working memory resulting in 
long-term memory encoding (Raaijmakers & Shifffin, 2004). Elaborative rehearsal also plays a role 
during extemalization as knowledge is elaborated during the process of transforming knowledge into 
externally represented information. 

Instructional methods therefore can be productively characterized in terms of presentation of infomiation 
and specification of learning activities. During infomiation presentation, infomiation is selected and 
organized to promote internalization of targeted knowledge. Learning activities have students express 
information in ways thought to promote targeted learning. The concept of “active learning” is the 
assertion that instructors can most influence learning by designing appropriate learning activities which 
require deeper processing. Thus, differences in learning activities can be expected to be a key 
differentiator of learning methods. This perspective suggests that the differences of typical learning 
activities used in each instructional environment are the major factors distinguishing instructional 
effectiveness of each environment. 

Clark (1994) has asserted that information delivery media, as mere conveyors of information, cannot 
affect formal learning. Clark suggested that research studies finding an advantage for a particular delivery 
medium did so because the studies did not compare equivalent instructional methods. Meta-analyses have 
tended to support this position (Bernard et al., 2004; Means, Toyama, Murphy, Bakia, & Jones, 2010; 
Sitzmann, Kraiger, Stewart, & Wisher, 2006). Means et al. found a statistically significant 0.24 effect size 
advantage for online and hybrid courses compared to classroom courses. But, the authors issued many 
cautions about their findings, including the fact that their meta-analysis included many studies that did not 
use the same instructional methods. These analyses suggest that differences in the effectiveness of 
instructional environments depend on differences in the instructional methods used in each environment, 
not on media differences. 

The intemalization-extemalization model can be extended to group learning by viewing social interaction 
(e.g., discussion) as a means for students to receive divergent infomiation from other students, and also as 
a means for students to express knowledge relevant to the learning task (Jorczak, 2011). Both learning 
processes are supported via group discussion that promotes both divergent thinking and elaborative 
rehearsal. The information intemalization-extemalization model provides a basis for the social 
constructivist concept of mutual knowledge construction by social interaction within learning groups. 


Table 1 

Exam Score Descriptive Statistics by Condition 


Instructional 

Environment 

n 

M 

SD 

SE(M) 

Classroom 

35 

.67 

.09 

.015 

Online 

69 

.74 

.11 

.013 


Note: Means (M) represent proportion of items answered correct across both exams. SD = standard 
deviation; SE = standard error. 
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A. Differences in Instructional Methods of Online and Classroom 
Environments 

The major difference between classroom and online environments is physical presence and 
synchronicity—classroom environments specify that the instructor and students all be present at the same 
location and time, while online environments separate instructor and students in location and time. 
Classroom environments include an information presentation method that is not available to online 
courses: face-to-face real time speech. Classroom environments are designed to support and promote this 
presentation method. Online environments cannot, by definition, use this presentation method. 

To compare instructional delivery environments and better understand how aspects of those environments 
affect learning, researchers must look beyond media and consider differences in instructional methods 
determined or influenced by the nature of the instructional delivery environment. A major difference in 
instructional method typically found in classroom and online environments involves the synchronicity of 
the instructional method (the degree to which students experience instruction simultaneously). While both 
environments can support synchronous and asynchronous learning methods, each displays a preference 
for one type. Classroom courses tend to use synchronous presentations and activities. Online courses tend 
to use asynchronous methods (in which students work whenever they choose and instructors interact with 
students when they choose). Online courses can present information to students synchronously, for 
example by web-casting a lecture or video to all students at the same time, but this approach is rarely used 
as it is difficult to implement and does not reflect the flexibility offered by the environment. Classroom 
courses also can have students do activities (including small group discussion) asynchronously. 

B. Interaction and Online Learning 

Classroom and online courses also tend to differ in the type of interaction they promote. A meta-analysis 
by Bernard et al. (2009) compared interactions in online environments by categorizing interaction into 
three treatment types: student-content (SC), student-teacher (ST), and student-student (SS). The meta¬ 
analysis compared studies that used these types of interaction as well as differences in the “strength” of 
the interactions. Bernard et al. (2009) found that online ST interactions did not affect learning to the same 
degree as SC or SS interactions. This result supports the idea that online discussion (which is primarily 
SS) has a greater effect on learning than teacher-led discussions (which are primarily ST). Classrooms are 
structured to promote ST interactions, including lecture-discussions. The meta-analysis results suggest a 
reason for possible differences between student performance in online and classroom environments. The 
interaction treatment type is an interesting variable, but the categories are broad and ignore important 
aspects of interaction (e.g., whether it is one-way or two way and the synchronicity of the discussion), so 
further refinement of the categorization of interaction is required. 

C. Asynchronous Discussion in Support of Learning 

Peer-to-peer guided discussion is an important collaborative learning technique, which has been 
repeatedly shown to be instructionally effective (Andriessen, Baker & Suthers, 2003; Garrison, Anderson, 
& Archer, 2000). How the method is implemented, which media are used and what instructional methods 
are used, tend to differ between classroom and online environments. Classroom instructors usually 
employ synchronous oral lecture-discussion. Online class discussion tends to be done asynchronously via 
text in “forums” in which students post messages and responses. 

Researchers have suggested learning advantages for asynchronous discussion (Garrison et al., 2000; 
Lapadat, 2002). Asynchronous discussion can be read at whatever pace aids reader understanding, and 
information can be rescanned and reviewed. Students can take as much time as they wish to respond, so 
discussion occurs at a much slower pace allowing for more reflection and processing than stream-of- 
consciousness speech (Garrison et al., 2000; Lapadat, 2002). No single student has the “floor” to speak, 
and no time limits are set by either outside constraints or limited attention spans. Knowledge and opinion 
that is externalized via text requires more effort and cognitive processing than speech. This extra 
processing improves externalized information and aids organization and specification of knowledge 
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(Newman, Webb, & Cochrane, 1997). Information in asynchronous discussion tends to diverge more 
(Newman et al., 1997), which aids learning (Jorczak & Bart, 2009). All students have access to the 
discussion and all students can contribute as much as they like. 

D. Research Question 

This study adds to the online versus classroom research literature by comparing student performance in 
online and classroom sections in which most instructional variables were kept constant, with one 
exception. In this in situ study, the key difference in instructional method is a difference between the 
discussion learning activities, though some differences in information presentation are also present. The 
overall goal of this analysis is to seek evidence about whether differences in instructional methods typical 
of classroom and online courses (i.e., synchronous lecture-discussion versus asynchronous small group 
discussion) are associated with differences in student exam performance. It is hypothesized that online 
students using asynchronous discussion will have better exam scores due to the instructional advantages 
of text-based asynchronous discussion. 

III. METHOD 


A. Participants and Procedures 

Participants were 104 college students (mostly first year students) enrolled in an introductory psychology 
course at a medium-size Midwestern public university. The course subject was introductory level 
psychology, and almost all students were required by their program to take the course. Students were 
enrolled in one of three course sections: a classroom section meeting for one hour three times a week (n = 
35), and two online sections delivered via a course management system (n = 69). 

The instructional design for both the classroom and online sections was nearly identical. The classroom 
and online courses covered identical material following the same sequence as well as the same schedule. 
The same instructor taught all three sections. Every section used the same materials including the same 
textbook, recorded videos, as well as supplemental materials (e.g., articles, handouts). Learning activities, 
such as written assignments, were the same for all three sections with one exception—student discussion. 

The instructional method of the online courses differed from the classroom course in information 
presentation and learning activities. In the classroom section, information was presented to students via 
lectures by the instructor, prior to in-class discussion. Classroom students also received copies of lecture 
slides on which they could take notes. Online students, on the other hand, received brief text-based 
“lectures” covering the same course content. 

Learning activities also differed between the two formats. Classroom students participated in both 
instructor-led and small-group discussion that was synchronous, face-to-face, and via speech; while 
students in the online sections used asynchronous peer discussion. Online students were graded for 
discussion participation, while classroom students were not. In short, the classroom course in this study 
used synchronous lecture-discussion and some synchronous small group discussion, but the online 
courses used asynchronous small group text discussion as a key instructional activity. 

B. Research Design and Data Analysis 

This study is quasi-experimental in design. Students were not randomly assigned to sections. The 
independent variable is the class delivery environment (online or classroom), and the primary dependent 
variable is the sum of course exam scores. Students in all sections were given two 50-item exams 
intended to assess knowledge acquisition and concept formation. These exams were timed and used 
multiple-choice items that were scored for accuracy (i.e., correct/incorrect). Exams were delivered to all 
sections via the university’s online learning management system. Overall course letter grades were 
assigned based on the total sum of learning activity points, weekly quiz results, exam scores, research 
participation points, and discussion participation (online only). Pearson product moment correlations 
were used to examine the relations between variables; independent-samples t-tests were used to examine 
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the mean difference in exam scores between course sections. Effect sizes were calculated following 
Cohen’s d. 


IV. RESULTS 

Across all exams, the mean proportion correct for classroom students was 0.67 and for the online students 
0.74, which results in a mean difference of 0.08 (see Table 1). The online students did statistically 
significantly better on the exams, t(102) = 3.56, p = .001. The effect size was d = 0.66, which suggests 
that an average student who participates in online discussion will score 25 percentile points higher on 
course exams than an average student who participates in classroom discussion. This effect is moderate to 
large, suggesting that a difference in the classroom and online environments has an important effect on 
exam performance. 

In the online sections, the correlation between exam scores and online discussion participation points was 
0.36 (p = .002), while the correlation between exam scores and all other (non-discussion) learning 
activities was 0.37 (p = .002). This result suggests a moderate association between online discussion 
participation and exam performance, as well as a moderate association between other learning activities 
and exam performance. For the total sample, the correlation between exam scores and total course points 
earned in learning activities (excluding online discussion participation points for the online sections) was 
0.30 (p = .01) suggesting that grades do not strictly follow assessed learning. Students with better test 
scores tended to get better grades, but the association is surprisingly weak. This result suggests that exam 
results were deemphasized as a determinant of grade in this course design. 

Students were not assigned to classes randomly, which damages any causal inferences drawn between the 
independent (discussion method) and dependent (exam scores) variables. The researchers were not able to 
access student information outside the course that would allow comparisons of the classroom and online 
students on variables of interest. An attempt was made to measure group equivalence by comparing group 
performance on the first course quiz. Quizzes were short multiple-choice tests based on chapter readings. 
Scores on the first quiz, before the course instructional design had time to take effect, were taken as an 
indicator of group knowledge and test taking skill prior to any effect of the course. No statistically 
significant difference was observed between the groups on the first quiz score (t(105) = 0.56, p = .572). 

V. DISCUSSION 

A Cohen’s d effect size of 0.66 is a moderate to large effect for an instructional intervention (Cohen, 
1977). Such a large difference in classroom versus online student performance is not something that is 
easily ignored or explained away. The observed difference in student exam performance is likely due to 
differences in the instructional methods. Because the instructional methods used in both sections are so 
similar, it is reasonable to conclude that the student performance gap stems from differences that did 
distinguish the classroom and online sections. While the methods differed in both manner of information 
presentation (lecture versus written information) and learning activities (synchronous oral versus 
asynchronous text discussion), prior research and theory point to differences in the learning activities as 
the likely primary cause of performance differences. 

Studies comparing oral versus text-based presentation of information have long suggested no difference 
in effect on student performance (Corey, 1934). In addition, both online and classroom students used the 
same textbook as an information source. We therefore argue that differences in information presentation 
between online and classroom sections is negligible for explaining differences in exam performance. 

Previous research conducted by Bernard et al., (2009) suggests that student-teacher interactions within 
classroom lecture-discussions may be partially responsible for the observed lower exam performance of 
classroom students as compared to the student-student interactions of online students in online discussion. 
The main difference in learning activities between the sections was synchronous oral teacher-led 
discussion (classroom) versus asynchronous textural peer discussion (online). We suggest that this 
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difference was primarily responsible for the observed difference in exam performance. The cognitive 
advantages of slowed processing, review ability, and equal access of asynchronous online test discussion 
appear to provide tangible learning benefits. Students in the online sections may retain information better 
because they choose when to attend to course information and assignments, as well as when to stop or 
rest. This learner control can contribute to better learning because students who voluntarily give attention, 
have the opportunity to direct their attention elsewhere, and return to the point at which there attention 
wandered, are less likely to miss important information. 

This study does not address aspects of effective discussion design but compares the basic functional 
differences of discussion typically found in online and classroom courses (i.e., synchronous speech versus 
asynchronous text). Other characteristics of asynchronous discussion design may also affect performance. 
We stress that the design of discussion in these classes met minimum quality standards that we take to be 
typical of current online classes and are typical of online courses that use peer discussion. 

A potentially important difference in discussion activities is that the online students were graded (i.e., 
earned points) for their participation in discussion (a common online practice). Classroom students were 
not assigned participation points and therefore may have had less incentive to participate. This distinction 
may be responsible for differences in learning due to discussion rather than whether the discussion is 
synchronous of not. This variable should be controlled in future studies of differences in discussion 
activities. 

If the difference in discussion activity is responsible for the difference in student exam performance, then 
online courses need to use and refine online asynchronous discussion in support of learning. In addition, 
classroom courses should move to hybrid designs that incoiporate asynchronous discussion activities with 
the aid of an online forum. 

A. Alternative Explanations 

Factors unrelated to differences in instructional methods between the sections could contribute to the 
observed effect. As mentioned, students were not randomly assigned to classroom and online sections. 
Therefore, one explanation that cannot be ruled out is that students who choose online courses tend to be 
more knowledgeable of course subject matter, are better students, have higher learning ability, are more 
motivated, and/or are better test-takers than their classroom counterparts. Many instructors offer an 
intuitive opinion that online students tend to be inferior to classroom students. Few studies have 
addressed this issue, and the ones that have show conflicting results regarding test performance and other 
variables between classroom and online students (Dutton, Dutton, & Perry, 2002; Kirtman, 2009) Such 
studies show that online and classroom students are more similar than different, but that online students 
report higher levels of interest, curiosity, and intrinsic motivation (Stevens & Switzer, 2006). Students in 
the classroom and online sections in this study were found to perform equally on the first quiz, suggesting 
that students did not differ in their knowledge or ability. 

Another possible explanation for the difference in test performance is instructor bias. The suggestion is 
that the instructor either prefers the online environment or has skills better suited to the online format. 
This explanation is unconvincing because most aspects of course design are not affected by the instructor 
during instruction, minimizing the effect of instructor bias in the two courses. The instructor does 
influence lecture-discussion, and to a lesser degree online discussion, but has minimum effect on all other 
course activities. 

Discrepancies in testing may account for some differences in exam performance. Exams were 
administered in a very similar way to students in both online and classroom sections. The exams were not 
“open book”, but students in both environments were allowed to create a one-page sheet of notes that they 
could refer to during the test. Exams were delivered via a learning management system (i.e., by computer) 
to all students, and were timed to limit opportunities for cheating. For the classroom section, the instructor 
was present during the administration of both exams which may have discouraged cheating compared to 
online sections, which were not monitored. 
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If discrepancies in testing are responsible for the observed difference in scores, then testing techniques for 
online exams must be re-evaluated and adjusted. Time limitations on tests should perhaps be decreased to 
reduce time available to seek answers from sources. The idea of an “open book” test that assesses higher 
order learning in which simple facts and definitions cannot be simply looked up may be more appropriate 
for online environments. The use of applications to “lock” browsers from accessing web pages other than 
the exam may prove useful for online testing (though students can simply use another device to access 
information). Clearly, more research comparing classroom and online testing is needed along with 
research about alternative online testing. 

B. Limitations 

This study most strictly generalizes to higher education, but there is no reason to expect similar results 
would not be found for other students if the result is due to differences in discussion activities. The major 
methodological limitation of this study is the lack of random assignment to groups as discussed above. 

The instructor for these courses is also the first author of this report, but the instructor did not know he 
would conduct this analysis prior to the completion of the course. As a result, no study conditions or any 
research design strategies were considered or established prior to the completion of the course. The idea 
to compare test scores came well after the course conclusion of the course. The fact that all classes had 
the same instructor reduces one potential confounding variable. Nevertheless, this study could be biased if 
the instructor had a superior ability and/or preference for online instruction. However, it is hard to see 
how instructor preference could result in such large differences in test scores given that most aspects of 
the instruction were identical. 

This study suggests more experimental work—as well as more theoretical work—is needed regarding 
asynchronous discussion. In future studies, the effect of discussion synchronicity can be more directly 
compared by implementing different discussion designs within the same delivery environment. If 
differences in online instructional methods are identified as being responsible for improved student 
performance, those methods may be able to be used in classroom or hybrid courses so that these 
instructional environments can take advantage of their added benefits. 
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